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The Effect of School Report Card Design on Usability, 
Understanding, and Satisfaction 


The Every Student Succeeds Act requires state education agencies to produce school report cards that present 
information about school characteristics and performance in a user-friendly way. The District of Columbia Office 
of the State Superintendent of Education partnered with the Regional Educational Laboratory Mid-Atlantic to test 
different school report card designs. 


The study compared five pairs of design alternatives: moving the link to the explanation of the School Transparency 
and Reporting (STAR) score from the top ribbon to under the reported score, depicting the points possible for each 
STAR metric instead of its floor and target scores, displaying the DC average proficiency as its own bar rather than 
as a line superimposed over the school’s score, reporting the year-over-year change in school performance rather 
than reporting only performance for each year, and marking a school’s offerings on a list of all possible offerings 
rather than reporting only what that the school offers. Some 824 participants viewed one of 32 school report card 
designs representing every possible combination of the tested design choices. Participants then answered factual 
questions about the information presented and reported how usable the site felt, how easy it was to find informa- 
tion, and whether they would recommend the site to others. 


Key findings 


e Design choices matter. Some designs had positive or neutral effects on all outcome measures, and others had 
mixed effects (see table). Effects of different choices are cumulative, and the best combination of design ele- 
ments improved understanding by 6 percent and willingness to recommend the site by half a scale point. Poli- 
cymakers should decide which outcomes are critical and evaluate report card designs against metrics for those 
designs. 


¢ Treatment effects were similar across user subgroups. Though subgroups of participants (such as mobile 
versus desktop users) differed in average survey responses, the effect of treatment factors was similar across 
subgroups. Designers can use readily available samples to gather input about designs but should focus on the 
relative rather than absolute performance of design alternatives for usability measures. 


Effect of proposed design changes on outcomes 


Proposed design change 


Display year-over- 


Change position Display STAR Put district average year change in List all potential 
Outcome measure of STAR link points earned in its own bar school performance offerings 
Usability A 1.5% Vv 3.5% 
Ease of finding specific information V 0.3% V 1.0% 
Understanding A 1.0% A 1.0% A 2.0% 
Willingness to recommend A 0.10 Vv 0.19 Vv 0.11 


STAR is School Transparency and Reporting. 


Note: Estimates of differences are reported only if there is a 70 percent chance that they are different than O. Usability and ease of finding specific 
information are the difference in the proportion of respondents who at least slightly agreed that the report card information was usable or easy to find. 
Understanding is the difference in the percentage of factual items answered correctly. Willingness to recommend is difference in scale points. 


Source: Authors’ analysis of data collected for this study (see appendix B). 


REL 2021-101 For the full report with technical details, see https://go.usa.gov/x6tCt. 


